Click Here!
home account info subscribe login search My ITKnowledge FAQ/help site map contact us


 
Brief Full
 Advanced
      Search
 Search Tips
To access the contents, click the chapter and section titles.

Oracle Performance Tuning and Optimization
(Publisher: Macmillan Computer Publishing)
Author(s): Edward Whalen
ISBN: 067230886x
Publication Date: 04/01/96

Bookmark It

Search this book:
 
Previous Table of Contents Next


Custom Benchmarks

Custom benchmarks are designed to create a workload that reflects the workload your user community will be generating. It is often difficult to predict the workload of hundreds or perhaps thousands of users, each with their own agenda. Although most custom benchmarks are designed by the end-user for a particular installation, there are some prepackaged custom benchmarks designed to generate a workload for a specific application.

If you are purchasing a packaged application, see whether there is a workload generator already built for that product. If there is, you can save yourself months of work designing and implementing such a workload generator. If one is provided, make sure that there is enough variance in it to allow customization for your installation.

Your custom benchmark should be designed to allow you to simulate the load of the number of users you support in production. It should also provide timing information that is critical for response time and throughput analysis. If your installation is required to support 1,000 concurrent users with 2-second response time criteria, it is important that the benchmark be able to simulate this.

When you put your system into production, it is too late to determine that you cannot support the required number of users, or that a checkpoint brings the system to its knees. Test these elements before the system rolls out into production.

If you are implementing a client/server system, make sure that you simulate the load over a network so that you can test the entire system. If possible, your benchmark should be an end-to-end test, simulating not just the workload but the display processing as well. Several vendors on the market today sell keystroke generators for character-mode applications as well as client/server load generators. Products such as Empower from Performix can be used to simulate thousands of users relatively easily.

Writing Your Own Benchmark

Writing your own benchmark is by far the best way to analyze the performance of a particular platform as well as to assist in the development and tuning phases of your application implementation. As in any project, writing a benchmark consists of the design, implementation, and analysis phases—each of which has its own special concerns. Of particular importance is to make sure that the proper workload is generated.

A custom benchmark can consist of an end-to-end test in which user keystrokes are emulated and data output examined or it can consist only of generating an equivalent workload on the server. Although the end-to-end test much more accurately represents the final workload, such a test is often too difficult to engineer. It is especially difficult to emulate a large number of PCs in a client/server configuration—but products are emerging on the market designed to help with this problem (consider Empower C/S from Performix).

These benchmarking/testing products are designed to capture a single transaction and let you programatically simulate hundreds and perhaps thousands of similar connections from a set of driving machines. These products allow you to randomize inputs and manage individual user contexts while simulating the load.

If you are just implementing a server workload test, be sure to carefully test the capacity of network connections and user contexts to ensure that your product roll out is not a failure. A robust server that can handle an intense workload does not guarantee that the configuration can handle hundreds or thousands of user connections. These factors are important to keep in mind during the design stage.

Design

The design stage is perhaps the most important phase in the development of a custom benchmark. Creating a workload that does not represent the load generated by the user community does not result in a successful implementation. In designing a custom benchmark, consider the following factors:

  The workload should match that of the user community that will be using the final implementation.
  Specify response time criteria.
  Build in other essential functions such as backups and other administrative tasks.
  If possible, generate the same network load as the final system.
  Take into account peak loads such as shift changes.
  Add a sufficient amount of randomness to vary the workload.
  If possible, use the actual application that will be used in the final configuration.
  Account for growth in the number of users and the work generated by them.

These are a few ideas you should consider in your custom benchmark. After completion of the design phase, you will want to implement the benchmark in the most efficient way possible.

Implementation

During the implementation phase, the benchmark is developed and run on the benchmark configuration. During this phase, it is important to take careful notes on the progress of the benchmark. Any changes made in the design or implementation should be noted.

It is here when the benchmark is actually run. The performance data is retrieved and analyzed. Be sure to collect as much data as you can to ensure that the analysis is complete. Look for data on such things as these:

  CPU utilization: Be especially aware of idle CPU cycles.
  Disk I/O: Look for hot spots in certain tables and disks.
  Network bandwidth: Don’t exceed the network limitations.
  Response times of transactions: It is important to make sure that you can meet the required response times.
  Transaction throughput: How much work can really get done?
  Is the benchmark operating as specified? Check to make sure that you are really driving the system the way you designed the benchmark to do.


Previous Table of Contents Next


Products |  Contact Us |  About Us |  Privacy  |  Ad Info  |  Home

Use of this site is subject to certain Terms & Conditions, Copyright © 1996-2000 EarthWeb Inc.
All rights reserved. Reproduction whole or in part in any form or medium without express written permission of EarthWeb is prohibited.